PromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers' Workflows

Prototyping AI applications is notoriously difficult. While large language model (LLM) prompting has dramatically lowered the barriers to AI prototyping, designers are still prototyping AI functionality and UI separately. We investigate how coupling prompt and UI design affects designers' workflows. Grounding this research, we developed PromptInfuser, a Figma plugin that enables users to create semi-functional mockups, by connecting UI elements to the inputs and outputs of prompts. In a study with 14 designers, we compare PromptInfuser to designers' current AI-prototyping workflow. PromptInfuser was perceived to be significantly more useful for communicating product ideas, more capable of producing prototypes that realistically represent the envisioned artifact, more efficient for prototyping, and more helpful for anticipating UI issues and technical constraints. PromptInfuser encouraged iteration over prompt and UI together, which helped designers identify UI and prompt incompatibilities and reflect upon their total solution. Together, these findings inform future systems for prototyping AI applications

Wait-learning: Leveraging conversational dead time for second language education

Second-language learners are often unable to find time for language practice due to constraints in their daily lives. In this paper, we examine how brief moments of waiting during a person's existing social conversations can be leveraged for second language practice, even if the conversation is exchanged in the first language. We present an instant messaging (IM) prototype, WaitChatter, that supports the notion of wait-learning by displaying contextually relevant foreign language vocabulary and micro-quizzes while the user awaits a response from her conversant. The foreign translations are displayed just-in-time in the context of the conversation to promote incidental learning. In a preliminary study of WaitChatter, we found that participants were able to integrate second language learning into their existing instant messaging activities, and that a particularly opportune time to embed foreign language elements may be immediately after the learner sends a chat message.Lincoln Laborator

The Design Space of Generative Models

Card et al.'s classic paper "The Design Space of Input Devices" established the value of design spaces as a tool for HCI analysis and invention. We posit that developing design spaces for emerging pre-trained, generative AI models is necessary for supporting their integration into human-centered systems and practices. We explore what it means to develop an AI model design space by proposing two design spaces relating to generative AI models: the first considers how HCI can impact generative models (i.e., interfaces for models) and the second considers how generative models can impact HCI (i.e., models as an HCI prototyping material)

ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principles

Large language model (LLM) prompting is a promising new approach for users to create and customize their own chatbots. However, current methods for steering a chatbot's outputs, such as prompt engineering and fine-tuning, do not support users in converting their natural feedback on the model's outputs to changes in the prompt or model. In this work, we explore how to enable users to interactively refine model outputs through their feedback, by helping them convert their feedback into a set of principles (i.e. a constitution) that dictate the model's behavior. From a formative study, we (1) found that users needed support converting their feedback into principles for the chatbot and (2) classified the different principle types desired by users. Inspired by these findings, we developed ConstitutionMaker, an interactive tool for converting user feedback into principles, to steer LLM-based chatbots. With ConstitutionMaker, users can provide either positive or negative feedback in natural language, select auto-generated feedback, or rewrite the chatbot's response; each mode of feedback automatically generates a principle that is inserted into the chatbot's prompt. In a user study with 14 participants, we compare ConstitutionMaker to an ablated version, where users write their own principles. With ConstitutionMaker, participants felt that their principles could better guide the chatbot, that they could more easily convert their feedback into principles, and that they could write principles more efficiently, with less mental demand. ConstitutionMaker helped users identify ways to improve the chatbot, formulate their intuitive responses to the model into feedback, and convert this feedback into specific and clear principles. Together, these findings inform future tools that support the interactive critiquing of LLM outputs

Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making

Machine learning (ML) is increasingly being used in image retrieval systems for medical decision making. One application of ML is to retrieve visually similar medical images from past patients (e.g. tissue from biopsies) to reference when making a medical decision with a new patient. However, no algorithm can perfectly capture an expert's ideal notion of similarity for every case: an image that is algorithmically determined to be similar may not be medically relevant to a doctor's specific diagnostic needs. In this paper, we identified the needs of pathologists when searching for similar images retrieved using a deep learning algorithm, and developed tools that empower users to cope with the search algorithm on-the-fly, communicating what types of similarity are most important at different moments in time. In two evaluations with pathologists, we found that these refinement tools increased the diagnostic utility of images found and increased user trust in the algorithm. The tools were preferred over a traditional interface, without a loss in diagnostic accuracy. We also observed that users adopted new strategies when using refinement tools, re-purposing them to test and understand the underlying algorithm and to disambiguate ML errors from their own errors. Taken together, these findings inform future human-ML collaborative systems for expert decision-making

Data-driven interaction techniques for improving navigation of educational videos

With an unprecedented scale of learners watching educational videos on online platforms such as MOOCs and YouTube, there is an opportunity to incorporate data generated from their interactions into the design of novel video interaction techniques. Interaction data has the potential to help not only instructors to improve their videos, but also to enrich the learning experience of educational video watchers. This paper explores the design space of data-driven interaction techniques for educational video navigation. We introduce a set of techniques that augment existing video interface widgets, including: a 2D video timeline with an embedded visualization of collective navigation traces; dynamic and non-linear timeline scrubbing; data-enhanced transcript search and keyword summary; automatic display of relevant still frames next to the video; and a visual summary representing points with high learner activity. To evaluate the feasibility of the techniques, we ran a laboratory user study with simulated learning tasks. Participants rated watching lecture videos with interaction data to be efficient and useful in completing the tasks. However, no significant differences were found in task performance, suggesting that interaction data may not always align with moment-by-moment information needs during the tasks.Engineering and Applied Science

Prognostic and Predictive Biomarkers in Patients With Coronavirus Disease 2019 Treated With Tocilizumab in a Randomized Controlled Trial

OBJECTIVES: To explore candidate prognostic and predictive biomarkers identified in retrospective observational studies (interleukin-6, C-reactive protein, lactate dehydrogenase, ferritin, lymphocytes, monocytes, neutrophils, d-dimer, and platelets) in patients with coronavirus disease 2019 pneumonia after treatment with tocilizumab, an anti-interleukin-6 receptor antibody, using data from the COVACTA trial in patients hospitalized with severe coronavirus disease 2019 pneumonia. DESIGN: Exploratory analysis from a multicenter, randomized, double-blind, placebo-controlled, phase 3 trial. SETTING: Hospitals in North America and Europe. PATIENTS: Adults hospitalized with severe coronavirus disease 2019 pneumonia receiving standard care. INTERVENTION: Randomly assigned 2:1 to IV tocilizumab 8 mg/kg or placebo. MEASUREMENTS AND MAIN RESULTS: Candidate biomarkers were measured in 295 patients in the tocilizumab arm and 142 patients in the placebo arm. Efficacy outcomes assessed were clinical status on a seven-category ordinal scale (1, discharge; 7, death), mortality, time to hospital discharge, and mechanical ventilation (if not receiving it at randomization) through day 28. Prognostic and predictive biomarkers were evaluated continuously with proportional odds, binomial or Fine-Gray models, and additional sensitivity analyses. Modeling in the placebo arm showed all candidate biomarkers except lactate dehydrogenase and d-dimer were strongly prognostic for day 28 clinical outcomes of mortality, mechanical ventilation, clinical status, and time to hospital discharge. Modeling in the tocilizumab arm showed a predictive value of ferritin for day 28 clinical outcomes of mortality (predictive interaction, p = 0.03), mechanical ventilation (predictive interaction, p = 0.01), and clinical status (predictive interaction, p = 0.02) compared with placebo. CONCLUSIONS: Multiple biomarkers prognostic for clinical outcomes were confirmed in COVACTA. Ferritin was identified as a predictive biomarker for the effects of tocilizumab in the COVACTA patient population; high ferritin levels were associated with better clinical outcomes for tocilizumab compared with placebo at day 28

Molecular Architecture of the Human Mediator–RNA Polymerase II–TFIIF Assembly

The authors perform a cryo-EM study of the 1.9 MDa human Mediator-RNA polymerase II-TFIIF assembly, which reveals the structural organization of the human transcription initiation apparatus

The Role of Protein Crystallography in Defining the Mechanisms of Biogenesis and Catalysis in Copper Amine Oxidase

Copper amine oxidases (CAOs) are a ubiquitous group of enzymes that catalyze the conversion of primary amines to aldehydes coupled to the reduction of O2 to H2O2. These enzymes utilize a wide range of substrates from methylamine to polypeptides. Changes in CAO activity are correlated with a variety of human diseases, including diabetes mellitus, Alzheimer’s disease, and inflammatory disorders. CAOs contain a cofactor, 2,4,5-trihydroxyphenylalanine quinone (TPQ), that is required for catalytic activity and synthesized through the post-translational modification of a tyrosine residue within the CAO polypeptide. TPQ generation is a self-processing event only requiring the addition of oxygen and Cu(II) to the apoCAO. Thus, the CAO active site supports two very different reactions: TPQ synthesis, and the two electron oxidation of primary amines. Crystal structures are available from bacterial through to human sources, and have given insight into substrate preference, stereospecificity, and structural changes during biogenesis and catalysis. In particular both these processes have been studied in crystallo through the addition of native substrates. These latter studies enable intermediates during physiological turnover to be directly visualized, and demonstrate the power of this relatively recent development in protein crystallography